Papers with consensus principle
EIT: Enhanced Interactive Transformer (2024.acl-long)
Copied to clipboard
| Challenge: | Existing multi-view learning models prioritize complementarity while ignoring consensus . EMHA allows for efficient modeling of global dependencies among tokens in parallel . |
| Approach: | They propose an enhanced multi-head self-attention (EMHA) that prioritizes complementarity while ignoring consensus. |
| Outcome: | The proposed method favors consensus among heads by introducing two models . it is superior on a wide range of language tasks with a modest increase in model size . |